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The purpose of this study is to evaluate the artificial intelligence-based distance 
education system called as ARTIMAT, which has been prepared in order to improve 
mathematical problem solving skills of the students, in terms of conceptual proficiency 
and ease of use with the opinions of teachers and students. 

The implementation has been performed with 4 teachers and 59 students in 10 th grade 
in an Anatolian High School in Trabzon. Many institutions and organizations in the 
world approach seriously to distance education besides traditional education. It is 
inevitable to use the distance education in teaching the problem solving skills in this 
different dimension of the education. In the studies in Turkey and abroad in the field of 
mathematics teaching, problem solving skills are generally stated not to be at the 
desired level and often expressed to have difficulty in teaching. 

For this reason, difficulties of the students in problem solving have initially been 
evaluated and the system has been prepared utilizing artificial intelligence algorithms 
according to the obtained results. In the evaluation of the findings obtained from the 
application, it has been concluded that the system is responsive to the needs of the 
students and is successful in general, but that conceptual changes should be made in 
order that students adapt to the system quickly. 
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INTRODUCTION 


The plenary aim of mathematics education is to bring in mathematical knowledge and 
skills that are required by daily life to the individual, to teach him problem solving and 
to bring in him a way of thinking that handles incidents including problem solving 
approach. For this reason, problem solving skills take important place among the 
mathematical skills (Baykul, 2004; De Corte, 2004). That problem solving keeps an 
important place in the overall objectives of mathematics course has carried this issue 
to the center of mathematics curriculum at multiple levels from primary school. Indeed, 
NCTM standards, as well, indicate that problem solving skills are needed to be primarily 
in mathematics teaching (NCTM, 2000). For the solution process of problems Polya 
(1957) recommends a framework that contains the stages of understanding the 
problem, selecting a strategy for the solution, the implementation of the strategy and 
the evaluation of the solution. g | 



Within this framework in many studies students are indicated to encounter a number of 
difficulties in understanding the concepts that problems include and the relationships 
between the concepts while they are solving problems (Ben-Hur, 2006; Chiu & Klassen, 
2008; Inzunza, 2006; Simon et al., 2004; Vicente, Orrantia & Verschaffel, 2007). 
According to Polya, one of the most important causes of these difficulties is that 
students do not perceive problem solving as a gradual process. This opinion is 
supported by some studies. 

Therefore, mathematics educators have a consensus on developing students' problem¬ 
solving skills and on issue that education should be prior aim (Cai, 2003; Karata§ & 
GLiven, 2004). For this reason, problem and the structure of problem solving, and 
increasing the success in problem solving is an issue studied by many educators and 
psychologists (Cai, 2003; Kilig & Samanci, 2005). 

In addition to many teaching methods, applications using computer technologies keep 
an important role in these studies. 

Now, many institutions and organizations in the world are seen to approach seriously 
to distance education besides traditional education. Especially after equipping 
universities with computer networks, it has been inevitable for the academic staff of 
universities to have interest for the distance education as distinct from the traditional 
education (Pala, 2006). Healing of technological infrastructure for internet-based 
distance education every day, eliminating of barriers of time and place for education, 
having opportunity to reach a wide audience and data's being continually updated has 
made internet-based education attractive (Kantar et al., 2008). 

It is emphasized in the new mathematics curriculum that student can construct his 
mathematical knowledge in accordance with the nature of constructivist approach, 
which is adopted by the program, using the software provided in an interactive way. 
Students should be encountered problems with different characteristics by analyzing 
problems that are primarily presented in the computer environment in the context of 
developing the problem solving skills. To accomplish this, software to enable student 
to live solution stages by presenting feedback according to the solution of the student 
is needed. Of course, programmability and availability of the software is also 
associated with the chosen topic. 

As computer does not have the possibility to think and make decisions in the 
proficiency of human, it is being usually worked on issues which are possible in terms 
of viability. Motion problems have an important place among the topics of mathematics 
curriculum related to problem solving due to that they contain questions of different 
types, the use of different solution ways and non-routine problems. Therefore, 
presentation of motion problems by an expert system in a way that they provide 
student to live problem solving processes will be a usable example in the field. 

"Expert Systems" is a software system that models the processes of judgment and 
decision-making of one or many people who are experts in a particular subject. Man 
acquires new knowledge throughout his life, deepens his views over time, changes and 
matures. Similarly, expert systems can broaden knowledge base and get rid of re¬ 
writing of the programs in the attachment of information. To be named a system as an 
expert system, it should have the abilities of detecting the errors of the user and 
directing the user by finding the mistakes (Nabiyev, 2005). 
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For this purpose, the present database of the expert system can be broadened 
according to the needs. In addition, the system can make logical deductions and it can 
be updated by field expert. In the light of given data, the purpose of this study is to 
evaluate the expert system called as ARTIMAT, which has been prepared in order to 
perform an education in which problem solving process gains importance to improve 
problem solving skills of the students and logic of problem solving is taught through a 
computer, in terms of conceptual proficiency and ease of use with the opinions of 
teachers and students. 

MATERIAL AND METHOD 

The Expert System Software: ARTIMAT 
General structure of ARTIMAT 

The general appearance of designed system is given in Figure: 1 The system has been 
designed in different modules for different types of users in it. 



Figure: 1 

General Structure of the System 


The system includes two different modules for two types of users, teacher and student. 
Different interfaces are opened in the entries of teacher and student in the system. 
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Figure: 2 Moodle Login Screen 
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In general; routing is done to teacher or student interfaces according to the user name 
and password in the entrance with the user name and password in the main screen 
prepared in Moodle environment. In teacher module, the user can add a new exercise 
or test problem to the system; can edit and remove former problems. In student 
module, the user can solve test exam using the education and exercises about motion 
problems. The schematic representations and explanations of the functioning of both of 
teacher and student modules are given below. 

Teacher Module 

Teacher module (Figure 3) is a module that user can add exercise and test problems, 
edit or remove former problems in the database when he logs in. 



Figure: 3 

The General Structure of Teacher Module 

The user firstly logins to the system through Moodle interface. The user who logins 
through Moodle interface summons the designed problem solving web page. Moodle 
forwards the user's information to the page. 



Figure: 4 

The Screen of Teacher Module 
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The problem that was recorded in teacher module is automatically solved by computer 
with a model using the methods of forward chaining and backward chaining that take 
an important place in artificial intelligence applications. Forward chaining is the 
application of inductive approach, towards the result from the data. Backward chaining 
is the application of deductive approach, directly towards the result. As a result, when 
the user in teacher module makes a problem input; 

> Problem is recorded to the Question Database. 

> The values of all variables in the problem are found and the user is asked 
to select the unknown -the result of the problem- 

> By the help of backward and forward chaining methods, the result of the 
problem and the solution ways are determined. 

> By transforming the result and solution ways to sub-problems step by 
step, they are recorded to the Sub-problem Database. 

> In this way, the system makes problems ready to ask the students. 

If the user wants to operate on a problem that was formerly entered into the Question 
Database, he can make necessary arrangements by summoning the related problem 
from the database this time and then the system makes the problem ready to ask 
students by following automatically the steps in problem input when the user confirms 
the problem. 

Student Module 

Student Module (Figure: 5) is the module in which the user can watch the lecture of the 
subject, can access the problem solving system or can have test exam when he logs in. 



Figure: 5 

General Structure of Student Module 
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Firstly, the user logs in to the system via Moodle interface (Figure: 6). 
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Figure: 6 

Student Module Moodle Screen 


The user who logs in to the system via Moodle interface summons the page of 
lecturing, designed problem solving web page or test exam page (Figure: 7). The data 
of the user who summons the page is forwarded into the page that pops up by Moodle. 



Figure: 7 

Problem Solving Screen 

The user can use lecturing page, or can enter problem solving system, or can use test 
exam independently of one another. If the user logs in to Problem Solving System; by 
starting to work initially with a moderate random question from the pool of questions 
that are grouped according to the level of difficulty, the system makes routings 
according to the level of the user. of. 


























The system makes this routing by asking sub-problems to the user for every step of the 
process and compares the answers of the user in each step with its own results. If the 
user gives correct answer for sub-problem, the system directs the user one at a time to 
the next step until finding the final result. 

If the answer is wrong, it wants the user to give a new answer alerting with a message 
that is appropriate for his answer. If the user gives wrong answer again, the system 
asks an easier question after being convinced that the question is too hard for the level 
of the user. The difficulty level of the questions increases as long as the user correctly 
knows the final results of the problems and decreases if the user gives wrong answers 
to them. The user can leave the system at any stage. All the steps of the students in 
problem solving whether they are right or wrong, time that is spent to solve the 
problem, access date to the system, the number of problems that they have solved 
wrong or right and total scores are kept in the system. 

Students do not have to necessarily follow the operation steps. Students can try 
forward steps in the way they want or, they can complete the solution by pressing "I 
FEEL LUCKY" button entering the answer to the area that is allotted at the beginning of 
the operation when they find the result or by entering the result at any time after the 
starting for the solution. 

RESEARCH MODEL 

Mixed research approach, in which qualitative and quantitative approaches were used 
together, has been preferred in the research (Cohen & Manion, 2000; Miles & 
Huberman, 1994). The opinions of both students and teachers have been taken in order 
to develop the expert system named ARTIMAT in terms of conceptual proficiency and 
ease of use in the study. However, academic successes of the students in 
implementation groups have been compared statistically by taking their mathematics 
examination marks for the first term of the academic year of 2011-2012. 

Performing the Implementation 

The implementation, which was conducted in order to evaluate the system, has been 
performed with 4 teachers and 59 students in 10 th grade in an Anatolian High School in 
Trabzon. ARTIMAT system has been implemented for both two groups for three weeks 
for two hours in each week in computer lab and in a way that each student has used 
his/her own computer. 

Data Collection Tools 

The data has been collected from implementation groups in three ways: success grades 
of mathematics course, teacher interviews and student interviews. Written interview 
forms have been prepared in order to get students' opinions and thoughts about the 
system. The questions asked in the interview forms are given below. 

> Which one of the features of the system did/didn't you liked most? 

> Did the system be helpful for your problem solving process? Can you 
explain? 

> Did you like the visual design of the system? What are your thoughts in 
terms of its being better? 

> Did the system change your point of view on problem solving? 
(Positive/negative) 


> What are the features of system the easiest to use? Can you explain? 

> Are there any sections that you have had difficulty to use the system? 
What are they? Can you explain? 

> What are your opinions and suggestions about the system? 

Teacher module and student module have been introduced to teachers in order to 
evaluate ARTIMAT system, and they have been provided to use it, and then their 
opinions have been taken with semi-structured interviews. Questions asked during the 
interview are as follows: 

> Which one of the features of the system did/didn't you liked most? 

> Do you think this system can help students develop their problem solving 

skills? 


> Can you compare the system with pencil and paper environment in terms 
of usability during problem solving process? 

> Do you evaluate the usefulness of the system during transferring the 
problem (during the writing of the problem) into computer environment? 

> Did you like the visual design of the system? What are your thoughts in 
terms of its being better? 

> Do you think that the system is successful in terms of finding the steps to 
follow in the solution of the problem? 

> Do you think that the system may be able to find all possible solution 
ways? 

> Do you think that the system can do a complete routing for student 
during problem solving? 

> What are your opinions and suggestions about the system? 

Analyzing the Data 

T-test has been implemented in order to see the academic successes and statuses 
according to each other of the students in implementation groups by taking their 
mathematics examination marks for the first term of the academic year of 2011-2012. 

By the help of this way, it has been checked whether their academic successes and 
gender factors in mathematics course were determinant. Interview data has been 
made transcript by the researcher. The obtained raw data has been initially transferred 
to writing in accordance with qualitative research techniques by reading one by one 
and then, the encodings have been made using expressions for determining the main 
theme of the opinions that the participant wanted to indicate. 

As a result of inferences from the encodings, the opinion, which participants wanted to 
indicate, has been reduced to a form that could be summarized with a few words and 
themes have been generated (Miles and Huberman, 1994). 

The data has been read at different times by different people for the reliability of the 
encoder and encoding, it has been detected that it was encoded similarly. Glesne and 
Peshkin (1992) reported that sharing the studies that were obtained from qualitative 
researches with people who were experts on qualitative researches and receiving 
feedback from them would increase the reliability of the research. 
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Also in this study, excerpts have been made from the opinions of students and teachers 
in order to increase the inner reliability and the validity of research findings. 

FINDINGS 

Mathematics Success Status of the Groups 

T-test results of the students in groups in the application according to their grades and 
genders are given in Table: 1 and Table: 2 by taking their mathematics examination 
marks for the first term of the academic year of 2011-2012. 

Table: 1 

T-test Results of the Students' Mathematics Exam Marks According to the Classes 



CLASS 

N 

Mean 

Std. 

Deviation 

Std. 

Error 

Mean 

P 

AVERAGES 

10-C 

30 

75,5067 

14,35460 

2,62078 

0,302 

10-A 

29 

71,5724 

14,65291 

2,72098 


As can be seen in Table 1, there is no significant difference between the averages of 
the classes (p>0.01) 


Table: 2 

T-test Results of the Students' Mathematics Exam Marks According to the Gender 


GENDER 

N 

Mean 

Std. 

Deviation 

Std. 

Error 

Mean 

P 

FEMALE 

47 

74,2043 

14,50856 

2,11629 


MALE 

12 

71,1000 

14,89539 

4,29993 

UplJ 


As can be seen in Table 2 as well, there is no significant difference between the 
averages of the genders (p>0.01). The results obtained have showed that groups had 
no differences according to their grades and genders. These findings show that 
interview data can be assessed without discrimination according to the grades and 
genders of the students. 

Student Interviews 

Each student's opinions and thoughts about the system, which they used for 6 hours in 
total, have been compiled without discriminating grade and gender and, results 
obtained from these have been given in the basis of interview questions below. 

Question: 1 

Which one of the features of the system did/didn't you like most? 

Features that were liked in students' answers have been determined as; 

> Providing individual learning 

> Being a more instructive and permanent system in mind 

> Providing the identification of the problem and knowing in-hands 

> Solving systematically the question step-by-step with a different method 

> Trying different solution ways by courtesy of the system 
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> Being easy to use 

> Visual design 

> Feature that students can add photos by creating their own profiles 

> Students' being able to communicate with each other via the system 

Features that were not liked in students' answers have been determined as; 

> Being unable to reach directly to the result 

> The obligation to follow the steps 

> Loosing time as there is a different solution way 

Question: 2 

Did the system be helpful for your problem solving process? Can you explain? 

Students have stated the positive sides of the system as follows; 

> It shows what should be done in the process of the problem solving 

> It helps students think on the solution of the problem 

> It increases the knowledge about the solution of the problems 

> It strengthens the feature of the judgment 

> It contributes to the understanding of the problem 

> It makes easier to solve the problem when it is got used to using the 
system 

> It warns when the wrong way of solution is selected 

> It develops the habit of systematic problem solving 

> Students have stated the negative sides of the system as follows; 

> It loses time as it prolongs the process of solution 

> Students are required to do operations, which they do by formula in 
short way, step by step in the system 

> They spend much more time for the questions that they will normally 
solve in less tim 

Question: 3 

Did you like the visual design of the system? 

What are your thoughts in terms of its being better? 

Students have stated in the positive opinions that; 

> The system is useful, can be reached quickly and conveniently and, its 
design is simple at the same time 

> System deficiencies that are wanted to be completed by the students are; 

> System should be enlivened with animations on the subject 

> Page color should be changed 

> More colorful mathematical symbols should be used in the system 

> Warnings for students should be more conspicuous 

Question: 4 

Did the system change your point of view on problem solving? (Positive/negative) 
Students have stated in their answers that; 

> They can see at which point they have made errors as they have solved 
the problems step by step 
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> It is provided to understand the question better to do operations, which 
they normally try to do in mind without writing the data, by writing in the 
system 


> Possibility of making a mistake decreases as they progress step by step to 
solve problems through the system 

> The system makes students to think over the problem instead of solving 
problems with formulas by heart 

> Solving problems step by step provides them to do operations more 
accurately 

> Solving problems in the computer environment makes problem solving 
process more enjoyable 

> Solving problems in the system provides a better learning although it 
takes longer 

> The system is useful for the development of human brain as it provides 
students the chance to use different solution ways for students in problem 
solving and, it can differentiate the monotonous types of question solving 

Question: 5 

What are the features of system the easiest to use? Can you explain? 

The majority of the students have reported that the system is easy to use in general. 

In addition, the following answers have come; 

> Given and desired, which are the first phase of the solution of the problem, 
are written 

> It is better seen in the table what the given and the desired are 

> Students can communicate with each other via the system 

> Problem solving steps can be selected 

Question: 6 

Are there any sections that you have had difficulty to use the system? 

What are they? Can you explain? 

Responses from students focus on certain points. It has great importance for students 

to use the system easily and efficiently. It is possible with the feedback to develop the 

system accurately and efficiently. 

Although it has been said that the system is generally easy to use according to the 

responses, the following difficulties have been indicated as follows; 

> Doing operations step by step seems complicated 

> I find it hard to choose the solution way 

> I have difficulty in transferring the operations to the computer 

> I find it hard to place the data 
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However, these have been stated by only five students but not in general. 

The following question has been asked to the students in the oral interviews with 
them. 


"What is the reason that you have determined as difficulties? Is the system complex or 

impractical?" 

The answers are as follows. 

Student 1: "In fact, the system is as easy when it is got used to but pen and paper is much easier 
for us. How many of us have solved questions via computer up to now? We are pencil and paper 

youth." 

Student 2: "The system could be more useful after a while, but it is unusual. I could not abandon 

to use pencil and paper ." 

Student 3: "No. The system is easy to use but pencil and paper is faster, we need to pay more 

attention on the computer ." 

Student 4: "The system does not allow us to pass when we write incomplete information. I have 

to step by step when I want to write the result directly." 

Student 5: "The system may initially seem useful, it is unnecessary to do operations step by step 

in some questions after a while." 


Question: 7 

What are your opinions and suggestions about the system? 

General answers of the students are as follows; 

> The system is successful and interesting. It makes the solution more fun, 

> The system has been crafted well, funny features relevant to mathematics 
such as games, jokes and etc. can be added, 

> The intelligence and skill games and music can be added to take mind off, 

> It should be developed for other types of problems such as pool, 
percentage and so on, 

> Actually, it is a nice system, but we attach importance to the result not to 
the phase. It will be better if it is result-oriented, 

> The system is nice, but phases should be facilitated, 

> The main page of the system can be better, 

> The system should include direct reaching to the solution 

Teacher Interviews 

As a result of interviews of teachers who use the system, the obtained themes are 
given in Table: 3. 

Four teachers who have participated in the interviews are coded as PI, P2, P3 and P4. 
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Table: 3 

Teachers' Opinions on the System 



PI 

P2 

P3 

P4 

Advantages of the system 

• Allowing to write 
the given Providing 
feedback at each 
stage (True/False) 

• Useful for 
beginners 

Routing the student 
Providing feedback 

olving the 

entered 

question 

Giving the 
opportunity to 
teacher and student 
to solve the problem 

Disadvantages of the 
system 

• Boring for students 
in advanced grade 

Visualization should 
be increased 
Causative feedback 
should be given 

Boring for 
students in 
advanced 
grade 

NI 

Status of helping the 
development of the 
problem solving skills 

• It helps 

It helps 

It helps to 
lower grade 
students 

It helps 

The usefulness of the 
system during transferring 
the problem into computer 
environment 

• Useful 

Useful 

Useful 

Useful 

Visual design of the system 

• It can be better 

Visualization can 
be further increased 

It can be 
better 

Classic, more 
formal 

Finding the steps to follow 
in the solution of the 
problem 

• Successful 

Partly successful 

Succes 

sful 

Successful 

Status of the system to 
route the students 

• Partly 
successful 

Routings are 
successful 

NI 

Routings are 
successful 

Suggestions 

• NI 

NI 

Causative 
feedback 
should be 
given 

it should be 
enriched with 
visuals It should 
be enriched with 
different types 
of guestions 


NI: No information 


In the interviews for the evaluation of the system, teachers state that the system is 
disadvantageous in some aspects while they find it advantageous in many aspects. 

It has been expressed as useful that the given and the desired were asked step by step 
for the problems in the system, and that feedback was provided as true-false at each 
stage, that a question, which was entered by teacher, was analyzed and solved by the 
computer, and that the opportunity to solve different types of questions was given to 
the students. However, disadvantages of the system have been expressed as that a 
causative feedback could not be given (information about why it is wrong), that the 
visualization was weak, and that it might be boring for the students in advanced 
degrees. The opinions of teachers on these answers have been presented below: 

"Requesting the given step by step and providing feedback at each stage is useful for 
beginner students to this subject... It is initially useful but may be tedious at later stages (PI)." 

"It is very useful to route the students and to say that the answer is right or wrong after 
solving the problem but, I think that visual part in the question should be at the forefront. The 
system should explain why it is wrong rather than saying that it is right or wrong (P2 )." 

"In fact, I liked the system... It can be a very useful program. It should be worked on some 
more. It is program that will help students at each stage even teacher if it becomes a program 
that can solve much more difficult questions after increasing the degrees of current 

questions (P4). ^3 




"I like its feature to solve the question. I dislike its feature to be boring for mid-level 
students. But it is a good system for lower level students who cannot learn (P3 )." 

When they have been asked to evaluate the usefulness of the system according to 
pencil and paper environment, two of teachers have stated that pencil and paper 
environment has been more useful. Other two teachers have stated that this might 
differ according to the ability of student to use computer. Excerpts for this opinion are 
as follows. 


"We learned that it would not be without pencil and paper. It was recorded to our 
subconscious like that. But I think it will work on the generations who have grown with the 
computer. Because, they use the computer like pencil and paper (P4 )." 

"Pencil and paper environment is handier. It takes time in the computer. For this reason, 
pencil and paper is more advantageous after learning the work (PI )." 

"I think that pencil and paper has an extra sense on human beings. I prefer pencil and 

paper rather than being on the screen (P3 )." 

"...it may not produce difficulty for students who use computer well (P2 )." 

The interface, which was used in transferring the problem into the system, has been 
found useful by all teachers. Visual design of the system has not been found satisfying 
by teachers and it has been determined that editing should have been done. 

The opinions about the visual design of the system have been expressed as follows: 

"Visual quality may be further improved. The main topics may be more noticeable. Each 
problem may be in a different color and its solution in a different color. (P2)." 

"I found it classic. I found it more formal. It may be more colorful. Today's students enjoy 
more striking, colorful and fun things. It may be supported with shapes (P4 )." 

"I did not like the visual design, that is, the selection of color. The blue color could have 
been better. Design is good except color. Headings may be given in different color. The main 

headings could have been a different color (P3 )." 

It has been seen that one teacher found the system partly successful in finding the 
steps to follow in the solution of a problem in the system while three teachers found it 
successful. The reason, why one teacher found partly successful, has been seen that 
the system could not resolve a type of question except the types of question that it 
solved. Guidance of the students by the system has been found successful by teachers. 
Teachers' opinions on this issue are as follows: 

"...I find the system successful in terms of that it allows working individually and sees own 

mistakes step by step (P4 )." 

"It is good for the child to solve the question with the guidance of the system. However, it 
may route the student better to say why he has made mistakes rather than saying that you have 

made mistake (PI )." 

"I think that it has made a good routing. At least, it provides systematically step-by-step 

solution. In this respect, it is good (P2 )." 
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It has been indicated that visual design should be healed and causative feedback 
should be given in general when teacher suggestions have been taken for the 
improvement of the system. Opinions on this issue are presented below. 

"... Let it show the solution of the question at the beginning or during the step rather than 

saying that you have made mistake (P3)." 

"Mathematics will be better when it is visually supported. The system should be visualized 

with shapes (P4)." 

CONCLUSION 

In this study, the evaluation- of the expert system called ARTIMAT, which was 
prepared to develop problem solving skills of students, in terms of conceptual 
proficiency and ease of use with teachers and students' opinions- has been done.By the 
help of the findings from student interviews, it has been concluded that students used 
all the features of the system and were satisfied with them, the system contributed to 
problem solving process in various aspects and, it developed students' points of view 
for problem solving in positive direction but they experienced time concern due to their 
habit of going directly to the result as they were exam-oriented. 

Although the students were reminded that the time was not a priority and that system 
was to eliminate the difficulties and shortcomings that they encountered in problem 
solving, time concern of students could not be eliminated. It has been concluded that 
the system did not change the students' habits of using pencil and paper, and that they 
could not give up the habit of using pencil and paper yet although they found the 
system successful in terms of learning the problem solving. Anyway, the study does not 
have an aim to eliminate completely the use of pencil and paper that students are 
familiar since then primary school. The majority of the students have indicated that the 
system was easy to use in general. 

It has been concluded from the findings obtained about the difficulties (which students 
encountered while using the system) that students were result-oriented and they 
found it hard to accept a different system as they solved test questions with paper and 
pencil for the examinations. By the help of findings from teacher opinions, teachers are 
seen to agree on the opinions that the system will be helpful for the development of 
problem solving skills of the students. 

In addition, some of the teachers have stated opinions on that this system will be more 
helpful on the lower grade students. By the help of responses, it has been concluded 
that the system is successful in general but it can be made more useful by doing some 
revisions in terms of design. In the light of findings and feedback from the students, 
visual design has been changed, and a more lively and attractive design has been 
prepared. In the system, a section -where intriguing information, anecdotes and stories 
in mathematical content and interesting events in the lives of famous mathematicians 
are described- is being prepared in order to keep students vigilant during problem 
solving process. Improvements on visual characteristics of the system are still being 
done. 
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